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TESTING  OR  FAULT-FINDING  FOR 
RELIABILITY  GROWTH: 

A  MISSILE  DESTRUCTIVE-TEST  EXAMPLE 

D.  P.  GAVER 

P.  A.  JACOBS 

Department  of  Operations  Research 
Naval  Postgraduate  School 
Monterey,  CA  93943 

ABSTRACT 

A  new  piece  of  equipment  has  been  purchased  in  a  lot  of  size  m. 

Some  of  the  items  can  be  used  in  destructive  testing  before  the  item 
is  put  into  use.  Testing  uncovers  faults  which  can  be  removed  from 
the  remaining  pieces  of  equipment  in  the  lot.  If  t  <  m  pieces  of 
equipment  are  tested,  then  those  that  remain,  mt  =  m-t,  have 
reduced  fault  incidence  and  are  more  reliable  than  initially,  but  mt 
may  be  too  small  to  be  useful,  or  than  is  desirable.  In  this  paper 
models  are  studied  to  address  this  question:  given  the  lot  size  m, 
how  to  optimize  by  choice  of  t  the  effectiveness  of  the  pieces  of 
equipment  remaining  after  the  test.  The  models  used  are  simplistic 
and  illustrative;  they  can  be  straightforwardly  improved. 

Key  words:  Reliability  growth;  Bayesian  sequential  analysis;  Poisson  process; 
destructive  testing;  how  much  testing  is  enough;  operational  testing 

1.  Problem  Setting 

A  new  piece  of  equipment  has  been  produced,  and  is  to  be  tested  before  being 
put  into  use.  An  example  is  a  military  missile.  Ultimate  testing  is  done 
destructively  by  firing  shots.  The  objective  is  to  send  equipment  to  the  field  with 


as  few  (design)  faults  as  possible,  so  testing  is  focused  on  finding  faults  and 
removing  them;  it  will  be  assumed  here  that  once  a  fault  is  discovered  it  can  be 
removed  by  change  of  design  or  componentry,  and  hence  that  a  mode  of  failure 
has  been  permanently  removed  from  all  remaining  missiles.  The  problem:  if 
missiles  are  bought  in  lots  of  m,  and  t<m  are  tested,  then  those  that  remain,  mt  = 
m-t,  have  reduced  fault  incidence  and  are  more  reliable  (the  lot  or  design  has 
experienced  "reliability  growth"),  but  mt  may  be  too  small  to  be  useful,  or  than  is 
desirable. 

We  address  two  problems. 

(a)  Given  the  lot  size,  m,  how  to  optimize  the  effectiveness  or  lethality  of  the 
missiles  remaining  after  t(<  m)  are  tested  by  choice  of  t; 

(b)  In  the  light  of  a  testing  program  of  length  f,  how  does  t  depend  upon  m;  or 
how  does  lot  size  affect  the  final  product's  quality,  where  quality  measures 
the  probability  of  overall  success  in  use?  This  means  that  both  reliability  and 
other  suitability  measures  are  combined  with  accuracy  and  target  destination 
probability  and  other  effectiveness  measures  to  obtain  an  overall  success 
probability  when  the  missile  is  fired.  The  focus  is  entirely  on  maximizing 
operational  capability,  given  the  lot  size,  m.  Other  calculations  can  be  made  to 
address  questions  of  final,  after-test,  missile  adequacy  to  meet  military  needs 
particularly  when  compared  to  alternative,  e.g.  currently  employed,  options. 
The  question  of  characterizing  the  uncertainty  with  which  such  a  comparison 
is  made  is  not  thoroughly  addressed  here. 

Related  issues  arise  in  reliability  growth  testing;  cf.  Ascher  and  Feingold 
(1984),  Balaban  (1978),  Barlow  and  Scheuer  (1966),  Barr  (1970),  Bhattacharyya  et 
al.  (1989),  Calabria  et  al.  (1992),  Fries  (1993),  Gross  et  al.  (1968),  Jayachadran  et  al. 
(1976),  Mazzuchi  et  al.  (1993),  Olsen  (1977),  Pollock  (1968),  and  Woods  (1990). 
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However,  in  traditional  reliability  growth  testing,  there  is  no  constraint  on  the 
number  of  tests  allowed. 

2.  Initial  Mathematical  Model 

Suppose  a  missile  design  initially  contains  Dq  potential  bugs  or  faults.  If 
present,  each  of  these  independently  inactivates  a  missile  flight  with  probability 
p,  or  does  not  operate  detrimentally  with  probability  1  -  p.  It  is  a  considerable 
simplification  to  assiune  that  p  is  the  same  for  all  fault/bug  types,  and  that  p  does 
not  depend  on  flight  time  or  other  conditions,  but  this  simplification  allows  a 
quick  initial  evaluation.  Note  that  if  m  missiles  are  built  as  described,  never 
tested  but  fired,  then  the  number.  So,  of  (later)  successful  flights  is,  given  Dq, 
distributed  binomially  with  probability  of  success  (1  -  p)^0;  consequently  its 
expectation  is 

E[So|Do]  =  sm(l-pfo,  (2.1) 

where  s  is  the  probability  that  a  missile  with  no  serious  faults  survives  and 
operates  properly.  Various  other  meaningful  measures  can  also  be  evaluated. 

2.1  Testing 

Suppose  t  missiles  are  test-fired.  If  some  fail  it  is  presumed  that  (a)  the 
particular  faults  causing  failure  are  identifiable,  and  (b)  that  they  are  successfully 
removed  from  the  remaining  missiles,  leaving  m-t  as  yet  unfired  and  potentially 
useful  in  actual  operations.  Furthermore,  these  are  now  more  reliable,  but  there  is 
obviously  a  tradeoff  involved  in  the  choice  of  t.  Thus  after  t  are  tested  (2.1)  turns 
into 

E[St\Dt]  =  s{m-t)(l-pf^  (2.2) 
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where  Df  is  the  number  of  potential  faults  remaining  after  t  test  firings.  It  is 
assumed  that  we  are  only  removing  single  "root-cause  faults"  that  can 
themselves  bring  about  missile  failure,  whereas  there  actually  could  be  a 
complicated  interlocking  sequence  of  fault  failures,  and  a  postmortem  could 
possibly  identify  them,  leading  to  their  simultaneous  remove.  This  optimistic 
situation  is  disregarded  here.  We  also  represent,  in  the  parameter  s,  the  influence 
of  non-removable  faults:  items  that  simply  fail  but  cannot  be  design-rectified. 
Existence  of  such  can  slow  down  the  reliability  growth  process  by  stimulating 
search  for  the  unattainable.  For  the  present  this  bit  of  realism  is  ignored,  as  is  the 
possibility  that  identification  of  a  removable  fault  leads  to  replacement  by  an 
item  of  higher  p-value  than  that  replaced!  The  present  model  is  optimistic  in  that 
a  new  item  is  essentially  compatible  with  s,  not  changing  it  by  much. 

2.2  Property  of  a  Test  of  Fixed  Length,  t 

In  order  to  choose  the  test  period,  t,  one  can  compute  the  expected  value  of 
those  that  survive  later  (active,  combat)  flights.  This  entails  removal  of  the 
condition  on  in  (2.2);  one  can  then  pick  the  f- value  so  as  to  maximize  that 
expectation.  This  is  one  answer  to  "how  much  is  enough  testing"  in  the  present 
context. 

Suppose  Do  bugs /faults  are  originally  present  and  we  ask  how  many  are 
present  after  time  f.  The  probability  that  any  one  is  still  present  is  (1  -  p)^;  by 
independence  Dt  is  binomial: 

p{d,  =A:|Do}  = 
with  generating  fxmction 

£[z'’<|Do]  = 


(^0 


Dn-fc 


(2.3) 


(2.4) 
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In  turn,  the  condition  on  Dq  can  be  removed;  if  (z)  is  the  generating  function 
of  Do  then 

In  Subsection  2.3  we  consider  Poisson-seeded  potential  faults.  In  Subsections 
2.5  and  2.6  we  consider  potential  faults  having  a  discrete  uniform  distribution 
and  a  discrete  uniform  distribution  with  a  random  range. 

2.3  Potential  Faults  are  Poisson-Seeded 

If  Do  is  assumed  Poisson  with  mean  A  then  directly  it  is  seen  that  Df  is  Poisson 
with  mean  A(1  -p)^,  which  has  generating  function 

e[z^‘  ]  =  (2.6) 

and  (2.2),  the  expected  number  of  successful  missions  after  testing  for  time  t 
(where  0  <  f  <  w): 

=  (2.7) 

Thus  if  all  parameters  (except  s)  are  known,  or  estimated,  we  can  discover  the 
value  of  f  =  toptOn)  that  maximizes  the  expected  number  of  missiles  sent  to  the  field 
that  will  function  properly  in  use.  Thus  we  have  an  initial  approach  to  a 
particular  problem  of  pre-determining  test  duration  so  as  to  "optimize"  a 
candidate  measure  of  mission  success. 

Note  that  the  distribution  of  Dq  can  be  regarded  as  a  Bayes  prior  on  an 
unknown  parameter.  Then  the  prior's  parameter.  A,  can  be  obtained  by 
combining  expert  judgment  and  data  on  previous  tested  and  fielded  comparable 
systems.  This  prior  can  be  updated  with  each  test  episode  using  Bayesian 
procedures.  This  approach  is  explored  in  Section  3. 
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2.4  A  Max-Min  Policy  for  Poisson-Seeded  Faults 

Suppose  nature  is  malevolent  and  for  any  number  of  tests  conducted  will 
choose  p  so  as  to  minimize  the  expected  number  of  successes  after  performing  f 
tests.  Let  s  =  1,  and  assume  Dq  is  Poisson  with  mean  X.  Let 


/(p)  =  lnE[S,] 

=  ln(m  -  f)  -  A(l  -  pfp. 


(2.8) 


Setting  ^  solving  for  p  results  in  the  minimizing  p,  pmin  =  1  /(I  +  f). 

For  this  value  of  p 


(2.9) 


A  criterion  to  choose  the  number  of  missiles  to  test  is  to  pick  the  number  of  tests, 
f,  that  maximizes  the  above.  We  will  call  this  policy  the  max-min  policy.  Such  a 
number  must  be  foxmd  numerically;  it  is  of  interest  to  compare  its  implications  to 
those  of  other  procedures. 


2.5  Alternate  Potential  Fault-Seeding  Distribution 

It  is  plausible  that  if  a  system  reaches  later  testing  stages  its  propensity  to 
contain  many  faults  is  low.  Perhaps  it  is  a  modification  of  a  previous  design  (an 
upgrade  in  military  parlance)  with  only  a  few  subsystems  being  candidates  for 
serious  faults.  In  this  case  the  Poisson  model,  which  admits  arbitrarily  many 
faults,  might  well  be  replaced  by  one  that  absolutely  limits  the  number  of  active 
faults,  so  we  investigate  one  of  the  simplest  alternatives:  a  discrete  uniform  for  Dq 
over  (0, 1, 2, . . .,  d).  Other  features  remain  as  before. 

The  generating  function  of  the  discrete  uniform  is 


(d  +  l)(l-z) 


(2.10) 
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so 


E[S(]  =  s(m-i) 


1 

d  +  1 


(2.11) 


For  numerical  illustration  we  match  means  to  that  of  the  Poisson:  djl  =  X ;  this 
will  not  always  be  possible  for  small  X  since  djx  >  1/2.  To  compare  the  expected 
number  of  successful  missions  after  testing  using  t  nussiles  for  the  Poisson  fault- 
seeding  model  and  the  discrete  uniform  fault-seeding  model  first  consider  the 
functions 


II 

1 

II 

d  1 
1 — a+- 

—a 

2 

2  2 

2 

^  J 

. 

(2.12) 


and 


id)-  ^ 

'  '  d  +  1 

a 

+ 1 


k=\ 


y  ^  y 


2  3! 


d[d-l) 


(2.13) 


where  a  =  (1  -  pYp.  Thus,  for  a  reasonably  small  a  =  (1  -  pYp,  the  expected  number 
of  successful  missions  after  testing  will  be  approximately  the  same  for  both 
models.  Examine  the  numerical  examples  to  follow  to  see  that  choice  of  the 
prior's  specific  form  may  be  of  secondary  effect. 


7 


2.6  Second  Alternative  for  Fault-Seeding:  Discrete  Uniform  with  Random 
Range 

Suppose  the  previous  setup  is  generalized  by  letting  d,  the  range  of  the 
uniform,  be  another  arbitrary  discrete  distribution,  denoted  {pjt;  fc  =  0,  1, 2, 
e.g.,  but  not  necessarily  Poisson.  From  (2.10) 


E\z^od]  =  ^ - 

L  d+ii-zl  ; 


(2.14) 


= lljl/pk^y  \lbifpk^y\ 

^  ^k=0- 


where  piw)  is  the  generating  function  of  {pitl- 
If  piw)  =  rMl  -  ®),  Poisson,  then  we  get 


L  J  p{l-z) 

Now  introduce  z  =  1  -  (1  -  p^p  as  before: 


>■  u(i-p)  pv  y 


L'  -  J  pil-pfpV 

In  order  to  match  means  it  is  easiest  to  calculate 


(2.15) 


(2.16) 


(2.17) 


E[Dop]  =  d/2  so  p  =  2E[Do].  (2.18) 

Thus,  substituting  (2.18)  into  (2.17)  for  E[Do]  =  djl  and  letting  a  =  pH  -p)t  results 


(2.19) 
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Comparing  (2.19)  and  (2.12),  it  is  seen  that  the  expected  number  of  successful 
missions  after  testing  for  the  Poisson  fault-testing  model  will  be  less  than  that  for 
the  discrete  uniform  with  Poisson  random  range. 

2.7  Numerical  Illustrations  and  Implications 

The  meaning  of  (2.7)  is  revealed  by  studying  some  special  cases.  Figures  1-2 
suggest  that  while  the  optimal  value  of  test  time  certainly  depends  upon  the 
parameter  values,  which  are  unknown  or  must  be  estimated,  the  optimum  values 
remain  in  a  relatively  narrow  range,  at  least  over  the  range  of  parameter  values 
studied.  For  what  seems  to  be  plausible  values  the  numbers  proposed  for  test  are 
a  smallish  fraction  of  lot  size,  m.  There  is  a  helpful  general  insight:  if  p,  the 
probability  of  fault  activation,  is  relatively  large  then  a  relatively  small  test  tends 
to  remove  many  potential  faults,  leaving  the  field  reliability  high,  whereas  a 
smaller  p-value  requires  somewhat,  but  not  substantially  many,  more,  since 
leaving  low-probability  offenders  in  place  is  relatively  undamaging.  The  max- 
min  policy  for  A  =  5  and  m  =  100  is  to  test  13  missiles  with  resulting  expected 
number  of  successes  75.9.  The  max-min  policy  for  A  =  5  and  m  -  500  is  to  test  29 
missiles  with  resulting  expected  number  of  successes  442.5.  Figures  1-2  show 
that  the  max-min  policy  is  (not  surprisingly)  somewhat  conservative. 

3.  Sequential  Destructive  Testing:  Myopic  Bayesian  Updating 

With  the  exception  of  the  max-min  analysis  given  in  Section  2.4,  the  previous 
analysis  assumes  that  the  design  defect  failure  probability,  p,  is  known,  or  at  least 
that  its  value  may  be  satisfactorily  approximated  off-line  from  data  for  analogous 
systems,  and  then  treated  as  "known".  Suppose,  however,  that  data  are  available 
sequentially  on  the  number  of  design  defects  that  were  revealed  on  an  initial  set 
of  f  e  (1, 2, ...)  test  firings  of  the  missile  in  question.  We  show  that  such  data  can 
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be  used  to  provide  a  sequentially  updated  inference  concerning  p,  and  thus  to 
decide  when  further  testing  is  not  justified.  In  Subsection  3.2  we  discuss  a 
criterion  which  compares  the  expected  number  of  successes  with  the  current 
posterior  distribution  of  p  with  that  if  we  look  forward  to  doing  one  more  test.  In 
Subsection  3.3  we  discuss  the  criterion  which  is  to  test  until  all  remaining 
(untested)  missiles  will  be  successful  with  a  preselected  probability.  The  problem 
we  discuss  is  related,  but  not  identical  to,  much  work  on  sequential  sampling  and 
decision  making.  See  in  particular  Chemoff  and  Ray  (1965),  and  Chemoff  (1966); 
Yang  et  al  (1982)  is  also  related. 

The  method  described  depends  on  these  factors  inherent  in  the  basic  model: 

Do  =  the  initial  number  of  design  defects  that  exist  in  the  missile  system. 

6  =  the  probability  a  fault  causes  a  failure  in  a  missile. 

Bi  -  the  number  of  faults  discovered  by  the  first  test.  Assume  all  the  faults 
are  repaired  upon  discovery. 

As  previously,  let  m  be  the  total  number  of  missiles. 

Assume 


P{Bl  =  ii|Do  =  4,9  =  p}  =  h=0 4; 


p{Do=do}  =  - 


dnl  ' 


do  =  0,1,. 


(3.1) 

(3.2) 


P{eedp}  =  f(p)dp. 

Then 


(3.3) 


P{d  e  dp,Bi  =  bi,Do  =  4}  =  f{p)- 


{do-bi)l 


dp.  (3.4) 


Let  Dj  =  Do  -  Bi,  the  number  of  remaining  faults;  then  from  (3.4)  it  follows  that 
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and 


p{eBdp,Bi  =i^,D,  =d,}  =  /(p)c-^  ^-A(l-p)  [^(1_ p)]  ’ 


p{s  e  dp,Di  =  rf,|Bi  =  li}  =  i:(6,)/(p)e-^(Ap)'^ 


dl  -A(l-p) 


where  K(bi)  = 
Similarly, 


1 

J 

LO 


1-1 


jf(p)e-^p^dp 


P^6edp,Bi  =  bi,B2  = 

&l!  fc2-' 


X... 


idu 


-dp 


-dp  (3.5) 


(3.6) 


h'  4’ 

where  Dj^  =  Dg  -  (B^  + . . .  +  Bj^),  the  ntimber  of  remaining  faults  after  k  tests. 


3.1  The  Expected  Number  of  Successes  after  t  Tests 

A  missile  is  called  a  success  if  no  faults  occur  during  its  launch  or  flight.  Let 
So  =  the  number  of  successful  missiles  if  no  testing  is  done;  (no  faults  are  fixed). 
Then 
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E[So]  =  £[£[So|Do,e]] 


=  mj  ^(l-p/‘‘e  ^^f{p)dp 

0<io=0  ‘*01 


-"^J  2.^  ^  ,  AW  (3.7) 

0dQ=0  "0- 

1 

=  mj  e~^f[p)dp. 

0 

Suppose  one  test  is  done  and  Bi  =  bi  faults  are  discovered  and  repaired;  let  Si 
be  the  number  of  successes  in  the  remaining  (m  - 1)  missiles. 

Odo=0  ^0- 

(3.8) 

=  (m-  1)J  e-^(^-’’>K(bi)f{py^(;ipf'  dp. 

0 

Similarly,  if  k  tests  are  conducted  and  Bi  faults  are  discovered  and  repaired  on 
the  z*  test,  the  expected  number  of  successes  in  the  remaining  (m  -  k)  missiles  is 

E[Sjt|Bi  =&!,... =&jt] 

\  -u,.  (3.9) 

=  {m-k)je  (  PI  PK{bi,...,bk)f{p;bi,...bk)dp 


where 


&l!  b2l  •"  fcfc! 
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3.2  The  Expected  Niimber  of  Successes  After  Looking  Forward  to  Doing  One 
More  Test 

Before  any  tests  are  conducted  consider  the  expected  number  of  successes  if 
one  test  were  conducted.  Let  Sj"  be  the  number  of  successes  using  the  remaining 
(m  - 1)  missiles.  From  (3.4) 


£[sf  ;B,  =b,eedp]  =  (m-  £(1  -pf  - '  ^ 

s=0 


(3.10) 


=  (m  -  l)f{p)- — i^exp{-A(l  -  p)p}dp. 


Thus, 


E^Si ;  0  €  dp]  =  (m  -  l)/(p)exp{-A(l  -  p)p}dp  (3.11) 

1 

E[sf  ]  =  (w  - 1)  J /(p)exp{-A{l  -  p)p}dp.  (3.12) 

0 

Suppose  k  tests  have  been  done  which  resulted  in  Bi  =  &i, ...,  Bjt  =  fcjt  faults 
being  discovered  and  repaired.  Consider  the  expected  number  of  successes  if  one 
more  test  were  conducted.  Let  be  the  number  of  remaining  successes  if 
another  test  is  conducted.  From  (3.6)  it  follows  that 


E^Sp,Bi=bi,B2=b2,...,Bi^=h,6edp^ 

=  {m-{k  +  l))f(p;bi,...,bk)dp  exp|-A(l - pfp^ 


where 


(3.15) 


bl!  b2!  *"  V 

E[Sfc  |Bi  =  bi,B2  =  ^2/”*/^ifc  =  ^it] 

1 

(m  -  (fc  + 1))  I  f{p;bi,. .  .,bfc)exp|-A(l  -  p)^pW  (3.15) 

_ _ 0 _ 

1 

J/(p;bl,...,bjt>fp 

0 

A  stopping  rule  might  be  to  stop  testing  at  ts  tests  where 

tg  =  min^fc ;  =  bj^j  >  Ej^Sj^^lBj  =  bj,..  .,Bj^  =  bj^j  +  c| 

where  C  is  a  constant  chosen  by  the  analyst;  possibly  C  =  0.  We  will  call  this  rule 
the  (myopic)  Bayes  rule. 

3.3  The  Probability  of  No  Failure  in  the  Remaining  Missile  Firings  After 
Conducting  t  Tests 

An  alternative  procedure  is  to  test  until  all  remaining  (imtested)  missiles  will 
be  successful  with  a  preselected  probability.  After  t  tests,  0  <  f  <  m,  the  proba¬ 
bility  aU  the  remaining  missiles  are  successes  is 


P{S„.,  =m-f}= 


(3.16) 


=  exp|-A(l-p/[l-(l-p)"'  *]| 


if  p  and  A.  are  known. 
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If  A  is  known  but  p  is  not  known,  then 


P{Sm-t  =  =  bi,Di  =  k,e  e  dp] 


=  f{P>hrh-A){'^-p)  ^ 


(3.17) 


kl 


dp 


where 


-vf 


Thus, 


and 


P{Sj„-t=rn-t,Bi=bi,...,Bi=bt,dedp] 
=  f{p-'Kh  • .  )exp|-A(l  -  1  -  (1  - 

P{Sffi-t=in-t\Bi=bi,...,Bi=bt} 

=  kJ  .  .,fcf  )exp|-A(l  -  p)^[l  -  (l  - 


where  K  = 


n-1 


LO 


A  rule  to  stop  testing  may  be  to  do  tp  tests  where 


(3.18) 


(3.19) 


tp  =  minjfc :  =  m  -  k\Bi  =  hj,. .  .,Bjt  =  >  aj  (3.20) 

where  a  =  0.8,  or  0.9,  etc. 

Numerical  integration  is  required  to  carry  out  the  above  procedures,  e.g.  to 
evaluate  integrals  in  (3.8),  (3.9),  (3.12),  (3.15)  and  elsewhere.  We  have  used 
Simpson's  rule  with  up  to  10*  order  difference  correction  for  a  step  size  h:0.0001 
(cf.  Hamming  (1973))  as  implemented  in  A  Graphical  Statistical  System,  AGSS. 
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3.4  Numerical  Examples 

Figure  3  presents  the  expected  number  of  successful  missile  flights  after 
having  conducted  t  tests  as  a  fimction  of  t  for  a  series  of  design  fault  discovery. 
There  are  three  faults.  One  fault  is  discovered  at  test  3;  one  at  test  4;  and  one  at 
test  6;  if  no  tests  are  conducted,  the  number  of  faults  discovered  is  0.  The  prior 
distribution  of  the  number  of  faults  at  time  0  is  assumed  to  be  a  Poisson 
distribution  with  mean  A  =  3.  The  prior  distribution  for  the  probability  of  fault 
discovery,  0,  is  uniform  over  [0, 1].  The  number  of  missiles  in  the  lot  m  =  25.  The 
solid  line  plots  the  expected  number  of  successes  with  no  additional  tests,  (3.7)  - 
(3.9).  The  dotted  line  plots  the  expected  number  of  successes  if  one  additional  test 
is  considered  (3.15).  The  dashed  line  plots  the  expected  number  of  successes  if  a 
fixed  number  of  tests  are  conducted  for  A  =  3  and  probability  of  discovery 
having  the  prior  distribution,  that  is,  from  (2.7), 

E[St]  =  (m  -  f)J  V(P)^P-  (3-21) 

0 

A  criterion  which  maximizes  the  expected  number  of  successes  for  a  fixed 
number  of  tests  would  stop  testing  after  4  tests.  A  criterion  which  stops  testing 
when  doing  one  more  test  would  not  result  in  a  larger  expected  number  of 
successes  would  also  stop  after  test  4.  Both  criteria  would  miss  the  one  fault  that 
does  not  appear  until  test  6.  The  max-min  policy  obtained  using  (2.9)  for  m  =  25 
and  A  =  3  would  also  test  4  missiles. 

Figure  4  displays  plots  of  the  probability  that  all  (m  - 1)  remaining  missiles  are 
successes  after  conducting  t  tests.  There  are  25  missiles  initially.  The  prior 
distribution  of  the  initial  number  of  faults  is  Poisson  with  mean  3.  The  prior 
distribution  for  the  discovery  probability  is  uniform  over  [0, 1].  The  solid  line 
displays  the  probability  of  all  remaining  missiles  being  successes  as  a  function  of 
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the  number  of  tests  using  the  same  fault  discovery  series  and  (3.19)  using  the 
posterior  distribution  of  the  discovery  probability.  The  dotted  line  is  the 
probability  of  all  remaining  missiles  being  successes  as  a  function  of  the  number 
of  tests  using  the  prior  distribution  of  the  discovery  probability  (fixed  number  of 
tests) 

P{St  =m-t}  =  j \{p)dp.  (3.22) 

0 

Consider  the  decision  rule  to  test  until  the  probability  that  all  remaining 
missiles  are  successes  is  at  least  y  for  y=  0.8.  For  the  uniform  prior,  the  fixed 
number  of  tests  calculation  would  test  6.  The  Bayes  calculation  would  test  12. 
Both  criteria  recommend  a  larger  number  of  tests  than  the  expected  number  of 
successes  criteria. 

4.  Discussion 

Our  model  directly  addresses  a  real  challenge  faced  by  the  testing 
community:  to  test  efficiently  with  operational  needs  in  mind.  The  present 
formulation  is  limited  and  simplified,  but  suggests  the  kinds  of  results  to  be 
expected,  and  that  can  be  practically  obtained.  In  particular  the  max-min 
approach  (Sec.  2.4)  provides  a  conservative  assessment  of  a  defensible 
conservative  number  of  tests  that  one  might  consider  making.  This  approach  is 
quite  robust  to  aspects  of  the  model  formulation  (it  actually  accommodates 
different  fault  failure  probabilities).  The  sequential  myopic  Bayes  approach 
(Sec.  3)  justifies  adjustment  of  test  effort  to  actual  data  obtained;  it  probably 
requires  further  detailed  development  before  being  practically  applicable,  but  the 
needed  modifications  are  understood,  and  are  being  made. 
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Implementation  of  the  present  approach  requires  a  certain  amount  of 
computing,  all  within  the  range  of  desktop  PCs  or  laptops.  It  is  likely  that  user- 
friendly  spreadsheet  realizations  of  the  cmrent  software  can  be  developed. 
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